Fine-grained Activities of People Worldwide

Every day, humans perform many closely related activities that involve subtle discriminative motions, such as putting on a shirt vs. putting on a jacket, or shaking hands vs. giving a high five. Activity recognition by ethical visual AI could provide insights into our patterns of daily life, however existing activity recognition datasets do not capture the massive diversity of these human activities around the world. To address this limitation, we introduce Collector, a free mobile app to record video while simultaneously annotating objects and activities of consented subjects. This new data collection platform was used to curate the Consented Activities of People (CAP) dataset, the first large-scale, fine-grained activity dataset of people worldwide. The CAP dataset contains 1.45M video clips of 512 fine grained activity labels of daily life, collected by 780 subjects in 33 countries. We provide activity classification and activity detection benchmarks for this dataset, and analyze baseline results to gain insight into how people around with world perform common activities. The dataset, benchmarks, evaluation tools, public leaderboards and mobile apps are available for use at visym.github.io/cap

A multimodal cell census and atlas of the mammalian primary motor cortex

ABSTRACT We report the generation of a multimodal cell census and atlas of the mammalian primary motor cortex (MOp or M1) as the initial product of the BRAIN Initiative Cell Census Network (BICCN). This was achieved by coordinated large-scale analyses of single-cell transcriptomes, chromatin accessibility, DNA methylomes, spatially resolved single-cell transcriptomes, morphological and electrophysiological properties, and cellular resolution input-output mapping, integrated through cross-modal computational analysis. Together, our results advance the collective knowledge and understanding of brain cell type organization: First, our study reveals a unified molecular genetic landscape of cortical cell types that congruently integrates their transcriptome, open chromatin and DNA methylation maps. Second, cross-species analysis achieves a unified taxonomy of transcriptomic types and their hierarchical organization that are conserved from mouse to marmoset and human. Third, cross-modal analysis provides compelling evidence for the epigenomic, transcriptomic, and gene regulatory basis of neuronal phenotypes such as their physiological and anatomical properties, demonstrating the biological validity and genomic underpinning of neuron types and subtypes. Fourth, in situ single-cell transcriptomics provides a spatially-resolved cell type atlas of the motor cortex. Fifth, integrated transcriptomic, epigenomic and anatomical analyses reveal the correspondence between neural circuits and transcriptomic cell types. We further present an extensive genetic toolset for targeting and fate mapping glutamatergic projection neuron types toward linking their developmental trajectory to their circuit function. Together, our results establish a unified and mechanistic framework of neuronal cell type organization that integrates multi-layered molecular genetic and spatial information with multi-faceted phenotypic properties

A multimodal cell census and atlas of the mammalian primary motor cortex

none258Here we report the generation of a multimodal cell census and atlas of the mammalian primary motor cortex as the initial product of the BRAIN Initiative Cell Census Network (BICCN). This was achieved by coordinated large-scale analyses of single-cell transcriptomes, chromatin accessibility, DNA methylomes, spatially resolved single-cell transcriptomes, morphological and electrophysiological properties and cellular resolution input-output mapping, integrated through cross-modal computational analysis. Our results advance the collective knowledge and understanding of brain cell-type organization1-5. First, our study reveals a unified molecular genetic landscape of cortical cell types that integrates their transcriptome, open chromatin and DNA methylation maps. Second, cross-species analysis achieves a consensus taxonomy of transcriptomic types and their hierarchical organization that is conserved from mouse to marmoset and human. Third, in situ single-cell transcriptomics provides a spatially resolved cell-type atlas of the motor cortex. Fourth, cross-modal analysis provides compelling evidence for the transcriptomic, epigenomic and gene regulatory basis of neuronal phenotypes such as their physiological and anatomical properties, demonstrating the biological validity and genomic underpinning of neuron types. We further present an extensive genetic toolset for targeting glutamatergic neuron types towards linking their molecular and developmental identity to their circuit function. Together, our results establish a unifying and mechanistic framework of neuronal cell-type organization that integrates multi-layered molecular genetic and spatial information with multi-faceted phenotypic properties.openCallaway, Edward M.; Dong, Hong-Wei; Ecker, Joseph R.; Hawrylycz, Michael J.; Huang, Z. Josh; Lein, Ed S.; Ngai, John; Osten, Pavel; Ren, Bing; Tolias, Andreas Savas; White, Owen; Zeng, Hongkui; Zhuang, Xiaowei; Ascoli, Giorgio A.; Behrens, M. Margarita; Chun, Jerold; Feng, Guoping; Gee, James C.; Ghosh, Satrajit S.; Halchenko, Yaroslav O.; Hertzano, Ronna; Lim, Byung Kook; Martone, Maryann E.; Ng, Lydia; Pachter, Lior; Ropelewski, Alexander J.; Tickle, Timothy L.; Yang, X. William; Zhang, Kun; Bakken, Trygve E.; Berens, Philipp; Daigle, Tanya L.; Harris, Julie A.; Jorstad, Nikolas L.; Kalmbach, Brian E.; Kobak, Dmitry; Li, Yang Eric; Liu, Hanqing; Matho, Katherine S.; Mukamel, Eran A.; Naeemi, Maitham; Scala, Federico; Tan, Pengcheng; Ting, Jonathan T.; Xie, Fangming; Zhang, Meng; Zhang, Zhuzhu; Zhou, Jingtian; Zingg, Brian; Armand, Ethan; Yao, Zizhen; Bertagnolli, Darren; Casper, Tamara; Crichton, Kirsten; Dee, Nick; Diep, Dinh; Ding, Song-Lin; Dong, Weixiu; Dougherty, Elizabeth L.; Fong, Olivia; Goldman, Melissa; Goldy, Jeff; Hodge, Rebecca D.; Hu, Lijuan; Keene, C. Dirk; Krienen, Fenna M.; Kroll, Matthew; Lake, Blue B.; Lathia, Kanan; Linnarsson, Sten; Liu, Christine S.; Macosko, Evan Z.; McCarroll, Steven A.; McMillen, Delissa; Nadaf, Naeem M.; Nguyen, Thuc Nghi; Palmer, Carter R.; Pham, Thanh; Plongthongkum, Nongluk; Reed, Nora M.; Regev, Aviv; Rimorin, Christine; Romanow, William J.; Savoia, Steven; Siletti, Kimberly; Smith, Kimberly; Sulc, Josef; Tasic, Bosiljka; Tieu, Michael; Torkelson, Amy; Tung, Herman; van Velthoven, Cindy T. J.; Vanderburg, Charles R.; Yanny, Anna Marie; Fang, Rongxin; Hou, Xiaomeng; Lucero, Jacinta D.; Osteen, Julia K.; Pinto-Duarte, Antonio; Poirion, Olivier; Preissl, Sebastian; Wang, Xinxin; Aldridge, Andrew I.; Bartlett, Anna; Boggeman, Lara; O’Connor, Carolyn; Castanon, Rosa G.; Chen, Huaming; Fitzpatrick, Conor; Luo, Chongyuan; Nery, Joseph R.; Nunn, Michael; Rivkin, Angeline C.; Tian, Wei; Dominguez, Bertha; Ito-Cole, Tony; Jacobs, Matthew; Jin, Xin; Lee, Cheng-Ta; Lee, Kuo-Fen; Miyazaki, Paula Assakura; Pang, Yan; Rashid, Mohammad; Smith, Jared B.; Vu, Minh; Williams, Elora; Biancalani, Tommaso; Booeshaghi, A. Sina; Crow, Megan; Dudoit, Sandrine; Fischer, Stephan; Gillis, Jesse; Hu, Qiwen; Kharchenko, Peter V.; Niu, Sheng-Yong; Ntranos, Vasilis; Purdom, Elizabeth; Risso, Davide; de Bézieux, Hector Roux; Somasundaram, Saroja; Street, Kelly; Svensson, Valentine; Vaishnav, Eeshit Dhaval; Van den Berge, Koen; Welch, Joshua D.; An, Xu; Bateup, Helen S.; Bowman, Ian; Chance, Rebecca K.; Foster, Nicholas N.; Galbavy, William; Gong, Hui; Gou, Lin; Hatfield, Joshua T.; Hintiryan, Houri; Hirokawa, Karla E.; Kim, Gukhan; Kramer, Daniel J.; Li, Anan; Li, Xiangning; Luo, Qingming; Muñoz-Castañeda, Rodrigo; Stafford, David A.; Feng, Zhao; Jia, Xueyan; Jiang, Shengdian; Jiang, Tao; Kuang, Xiuli; Larsen, Rachael; Lesnar, Phil; Li, Yaoyao; Li, Yuanyuan; Liu, Lijuan; Peng, Hanchuan; Qu, Lei; Ren, Miao; Ruan, Zongcai; Shen, Elise; Song, Yuanyuan; Wakeman, Wayne; Wang, Peng; Wang, Yimin; Wang, Yun; Yin, Lulu; Yuan, Jing; Zhao, Sujun; Zhao, Xuan; Narasimhan, Arun; Palaniswamy, Ramesh; Banerjee, Samik; Ding, Liya; Huilgol, Dhananjay; Huo, Bingxing; Kuo, Hsien-Chi; Laturnus, Sophie; Li, Xu; Mitra, Partha P.; Mizrachi, Judith; Wang, Quanxin; Xie, Peng; Xiong, Feng; Yu, Yang; Eichhorn, Stephen W.; Berg, Jim; Bernabucci, Matteo; Bernaerts, Yves; Cadwell, Cathryn René; Castro, Jesus Ramon; Dalley, Rachel; Hartmanis, Leonard; Horwitz, Gregory D.; Jiang, Xiaolong; Ko, Andrew L.; Miranda, Elanine; Mulherkar, Shalaka; Nicovich, Philip R.; Owen, Scott F.; Sandberg, Rickard; Sorensen, Staci A.; Tan, Zheng Huan; Allen, Shona; Hockemeyer, Dirk; Lee, Angus Y.; Veldman, Matthew B.; Adkins, Ricky S.; Ament, Seth A.; Bravo, Héctor Corrada; Carter, Robert; Chatterjee, Apaala; Colantuoni, Carlo; Crabtree, Jonathan; Creasy, Heather; Felix, Victor; Giglio, Michelle; Herb, Brian R.; Kancherla, Jayaram; Mahurkar, Anup; McCracken, Carrie; Nickel, Lance; Olley, Dustin; Orvis, Joshua; Schor, Michael; Hood, Greg; Dichter, Benjamin; Grauer, Michael; Helba, Brian; Bandrowski, Anita; Barkas, Nikolaos; Carlin, Benjamin; D’Orazi, Florence D.; Degatano, Kylee; Gillespie, Thomas H.; Khajouei, Farzaneh; Konwar, Kishori; Thompson, Carol; Kelly, Kathleen; Mok, Stephanie; Sunkin, SusanCallaway, Edward M.; Dong, Hong-Wei; Ecker, Joseph R.; Hawrylycz, Michael J.; Huang, Z. Josh; Lein, Ed S.; Ngai, John; Osten, Pavel; Ren, Bing; Tolias, Andreas Savas; White, Owen; Zeng, Hongkui; Zhuang, Xiaowei; Ascoli, Giorgio A.; Behrens, M. Margarita; Chun, Jerold; Feng, Guoping; Gee, James C.; Ghosh, Satrajit S.; Halchenko, Yaroslav O.; Hertzano, Ronna; Lim, Byung Kook; Martone, Maryann E.; Ng, Lydia; Pachter, Lior; Ropelewski, Alexander J.; Tickle, Timothy L.; Yang, X. William; Zhang, Kun; Bakken, Trygve E.; Berens, Philipp; Daigle, Tanya L.; Harris, Julie A.; Jorstad, Nikolas L.; Kalmbach, Brian E.; Kobak, Dmitry; Li, Yang Eric; Liu, Hanqing; Matho, Katherine S.; Mukamel, Eran A.; Naeemi, Maitham; Scala, Federico; Tan, Pengcheng; Ting, Jonathan T.; Xie, Fangming; Zhang, Meng; Zhang, Zhuzhu; Zhou, Jingtian; Zingg, Brian; Armand, Ethan; Yao, Zizhen; Bertagnolli, Darren; Casper, Tamara; Crichton, Kirsten; Dee, Nick; Diep, Dinh; Ding, Song-Lin; Dong, Weixiu; Dougherty, Elizabeth L.; Fong, Olivia; Goldman, Melissa; Goldy, Jeff; Hodge, Rebecca D.; Hu, Lijuan; Keene, C. Dirk; Krienen, Fenna M.; Kroll, Matthew; Lake, Blue B.; Lathia, Kanan; Linnarsson, Sten; Liu, Christine S.; Macosko, Evan Z.; Mccarroll, Steven A.; Mcmillen, Delissa; Nadaf, Naeem M.; Nguyen, Thuc Nghi; Palmer, Carter R.; Pham, Thanh; Plongthongkum, Nongluk; Reed, Nora M.; Regev, Aviv; Rimorin, Christine; Romanow, William J.; Savoia, Steven; Siletti, Kimberly; Smith, Kimberly; Sulc, Josef; Tasic, Bosiljka; Tieu, Michael; Torkelson, Amy; Tung, Herman; van Velthoven, Cindy T. J.; Vanderburg, Charles R.; Yanny, Anna Marie; Fang, Rongxin; Hou, Xiaomeng; Lucero, Jacinta D.; Osteen, Julia K.; Pinto-Duarte, Antonio; Poirion, Olivier; Preissl, Sebastian; Wang, Xinxin; Aldridge, Andrew I.; Bartlett, Anna; Boggeman, Lara; O’Connor, Carolyn; Castanon, Rosa G.; Chen, Huaming; Fitzpatrick, Conor; Luo, Chongyuan; Nery, Joseph R.; Nunn, Michael; Rivkin, Angeline C.; Tian, Wei; Dominguez, Bertha; Ito-Cole, Tony; Jacobs, Matthew; Jin, Xin; Lee, Cheng-Ta; Lee, Kuo-Fen; Miyazaki, Paula Assakura; Pang, Yan; Rashid, Mohammad; Smith, Jared B.; Vu, Minh; Williams, Elora; Biancalani, Tommaso; Booeshaghi, A. Sina; Crow, Megan; Dudoit, Sandrine; Fischer, Stephan; Gillis, Jesse; Hu, Qiwen; Kharchenko, Peter V.; Niu, Sheng-Yong; Ntranos, Vasilis; Purdom, Elizabeth; Risso, Davide; de Bézieux, Hector Roux; Somasundaram, Saroja; Street, Kelly; Svensson, Valentine; Vaishnav, Eeshit Dhaval; Van den Berge, Koen; Welch, Joshua D.; An, Xu; Bateup, Helen S.; Bowman, Ian; Chance, Rebecca K.; Foster, Nicholas N.; Galbavy, William; Gong, Hui; Gou, Lin; Hatfield, Joshua T.; Hintiryan, Houri; Hirokawa, Karla E.; Kim, Gukhan; Kramer, Daniel J.; Li, Anan; Li, Xiangning; Luo, Qingming; Muñoz-Castañeda, Rodrigo; Stafford, David A.; Feng, Zhao; Jia, Xueyan; Jiang, Shengdian; Jiang, Tao; Kuang, Xiuli; Larsen, Rachael; Lesnar, Phil; Li, Yaoyao; Li, Yuanyuan; Liu, Lijuan; Peng, Hanchuan; Qu, Lei; Ren, Miao; Ruan, Zongcai; Shen, Elise; Song, Yuanyuan; Wakeman, Wayne; Wang, Peng; Wang, Yimin; Wang, Yun; Yin, Lulu; Yuan, Jing; Zhao, Sujun; Zhao, Xuan; Narasimhan, Arun; Palaniswamy, Ramesh; Banerjee, Samik; Ding, Liya; Huilgol, Dhananjay; Huo, Bingxing; Kuo, Hsien-Chi; Laturnus, Sophie; Li, Xu; Mitra, Partha P.; Mizrachi, Judith; Wang, Quanxin; Xie, Peng; Xiong, Feng; Yu, Yang; Eichhorn, Stephen W.; Berg, Jim; Bernabucci, Matteo; Bernaerts, Yves; Cadwell, Cathryn René; Castro, Jesus Ramon; Dalley, Rachel; Hartmanis, Leonard; Horwitz, Gregory D.; Jiang, Xiaolong; Ko, Andrew L.; Miranda, Elanine; Mulherkar, Shalaka; Nicovich, Philip R.; Owen, Scott F.; Sandberg, Rickard; Sorensen, Staci A.; Tan, Zheng Huan; Allen, Shona; Hockemeyer, Dirk; Lee, Angus Y.; Veldman, Matthew B.; Adkins, Ricky S.; Ament, Seth A.; Bravo, Héctor Corrada; Carter, Robert; Chatterjee, Apaala; Colantuoni, Carlo; Crabtree, Jonathan; Creasy, Heather; Felix, Victor; Giglio, Michelle; Herb, Brian R.; Kancherla, Jayaram; Mahurkar, Anup; Mccracken, Carrie; Nickel, Lance; Olley, Dustin; Orvis, Joshua; Schor, Michael; Hood, Greg; Dichter, Benjamin; Grauer, Michael; Helba, Brian; Bandrowski, Anita; Barkas, Nikolaos; Carlin, Benjamin; D’Orazi, Florence D.; Degatano, Kylee; Gillespie, Thomas H.; Khajouei, Farzaneh; Konwar, Kishori; Thompson, Carol; Kelly, Kathleen; Mok, Stephanie; Sunkin, Susa